Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display

ABSTRACT

Systems, devices and methods that enable a user to access and interact with content displayed on a portable electronic display in an inconspicuous, hands-free manner are described. There is disclosed a completely wearable system comprising a wearable muscle interface device and a wearable head-mounted display, as well as methods for using the wearable system to effect interactions between the user and content displayed on the wearable head-mounted display. The wearable muscle interface device includes muscle activity sensors worn on an arm of the user to detect muscle activity generated when the user performs a physical gesture. The wearable system is adapted to recognize a plurality of gestures made by the user and, in response to each recognized gesture, to effect one or more interaction(s) with content displayed on the wearable head-mounted display.

BACKGROUND Technical Field

The present systems, devices, and methods relate generally to wearablemuscle interfaces, and more specifically to a wearable muscle interfacethat interacts with content displayed on a wearable head-mounteddisplay.

Description of the Related Art Wearable Electronic Devices

Electronic devices are commonplace throughout most of the world today.Advancements in integrated circuit technology have enabled thedevelopment of electronic devices that are sufficiently small andlightweight to be carried by the user. Such “portable” electronicdevices may include on-board power supplies (such as batteries or otherpower storage systems) and may be designed to operate without a physicalwire-connection to any stationary (i.e., non-portable) electronic system(except, in some cases, during charging).

The convenience afforded by the portability of electronic devices hasfostered a huge industry. Smartphones, audio players, laptop computers,tablet computers, and ebook readers are all examples of portableelectronic devices. However, the convenience of being able to carry aportable electronic device has also introduced the inconvenience ofhaving one's hand(s) encumbered by the device itself. This problem isaddressed by making an electronic device not only portable, butwearable.

A wearable electronic device is any portable electronic device that auser can carry without physically grasping, clutching, or otherwiseholding onto the device with their hand(s). For example, a wearableelectronic device may be attached or coupled to the user by a strap orstraps, a band or bands, a clip or clips, an adhesive, a pin and clasp,an article of clothing, tension or elastic support, an interference fit,an ergonomic form, etc. Examples of wearable electronic devices includedigital wristwatches, electronic armbands, electronic rings, electronicankle-bracelets or “anklets,” head-mounted electronic displays, hearingaids, and so on.

Human-Electronics Interfaces

A wearable electronic device may provide direct functionality for a user(such as audio playback, data display, measurement and monitoring,computing functions, “virtual reality,” “augmented reality,” etc.) or itmay provide electronics to interact with, communicate with, or controlanother electronic device. For example, a wearable electronic device mayinclude sensors that detect inputs affected by a user and transmitsignals to another electronic device based on those inputs. Sensor-typesand input-types may each take on a variety of forms, including but notlimited to: tactile sensors (e.g., buttons, switches, touchpads, orkeys) providing manual control, acoustic sensors providingvoice-control, electromyography sensors providing gesture control,and/or accelerometers providing gesture control.

A human-computer interface (“HCI”) is an example of a human-electronicsinterface.

Electromyography Devices

Electromyography (“EMG”) is a process for detecting and processing theelectrical signals generated by muscle activity. EMG devices employ EMGsensors that are responsive to the range of electrical potentials(typically μV-mV) involved in muscle activity. EMG signals may be usedin a wide variety of applications, including: medical monitoring anddiagnosis, muscle rehabilitation, exercise and training, prostheticcontrol, and even in controlling functions of electronic devices.

Human-electronics interfaces that employ EMG have been proposed. Forexample, U.S. Pat. Nos. 6,244,873 and 8,170,656 both describe proposalsin which a user dons a wearable EMG device and performs physicalgestures to control functions of a separate electronic device. In bothcases, the separate electronic device is not itself a wearableelectronic device, so true hands-free operation of and/or access to theseparate electronic device is not achieved. For example, both casesdescribe using EMG signals to control mobile phones, smart phones,computers, laptop computers, and so on, all of which still typicallyrequire the user to use their hand(s) to carry the device and/or toorient the device in such a way that the user may see, access, receivefeedback from, and/or generally interact with a display screen on thedevice.

Interacting with Head-Mounted Displays

As described above, portable electronic devices that include displayscreens typically require the user to use their hand(s) to carry thedevice and/or to orient the device so that the user may see, access,receive feedback from, and/or generally interact with the device'sdisplay screen. Occupying the user's hand(s) is an inconvenience thatcan significantly hinder the user's ability to interact with theportable electronic device and/or to interact with other aspects oftheir environment while operating the portable electronic device.However, this hindrance is at least partially overcome by making thedisplay screen of the portable electronic device wearable. Making thedisplay screen of the portable electronic device wearable enables theuser to see, access, and/or receive feedback from the display screenwithout using their hand(s). In recent years, wearable head-mounteddisplays have begun to gain wider acceptance, with a number of recentlyintroduced wearable head-mounted display devices having the potentialfor widespread adoption by consumers.

One such device disclosed in U.S. Pat. No. 8,203,502 issued to Chi etal. utilizes a finger operable input device such as a touch pad builtinto the wearable head-mounted display (e.g. built into a side-arm of apair of glasses, with one of the lenses functioning as a display screen)such that a user can interact with and control content appearing on thedisplay screen with positioning and movement of a finger along a surfaceof the input device. A potential drawback of this approach is that auser is required to conspicuously raise his or her hand to touch theinput device each time the user wants to interact with content displayedon the screen. Furthermore, even though the display itself is wearable,it is still controlled by touch and so is not actually hands-free (thusnegating part of the benefit of making the display wearable in the firstplace).

Another such device is disclosed in US 2012/0293548 (Perez et al.) inwhich a head-mounted display provides users with supplementalinformation on a display screen provided in at least one of the lensesof a pair of glasses. A processing unit may be connected to thehead-mounted display to provide the computing power necessary for itsoperation. However, the method of user interaction with the display isnot specified.

Yet another example of such a device is disclosed in U.S. Pat. No.8,212,859 issued to Tang et al. in which a source image is projectedonto screens built into head-mounted displays worn by a user. Tang etal. focuses on the method and system for projection, and does notspecify the manner of user interaction with the head-mounted displaydevice.

U.S. Pat. No. 5,482,051 ('051 patent) describes a human-electronicsinterface in which a user's EMG signals are detected and used tointeract with content that is ultimately displayed on a head-mountedvisual display unit. However, the interface described in the '051 patentis not a portable system. The human-electronics interface described inthe '051 patent consists of at least three disparate components that arecommunicatively coupled in series with one another; i) a set of EMGsensors, ii) a stand-alone processing system, and iii) a head-mountedvisual display unit. Although the set of EMG sensors and thehead-mounted visual display unit are both physically coupled to (i.e.,worn by) the user, there is no direct communication between the set ofEMG sensors and the head-mounted visual display unit. Detected EMGsignals are sent from the set of EMG sensors to the stand-aloneprocessing system (i.e., off the body of the user) where they areprocessed to achieve some effect, and then signals that represent theeffect are sent from the processing system to the head-mounted visualdisplay unit where the effect is displayed to the user. The stand-aloneprocessing system mediates all communication between the set of EMGsensors and the head-mounted visual display unit. The processing systemis not worn by the user and is not portable (i.e., it is stationary),and therefore the human-electronics interface described in the '051patent is limited in that the user must be in close proximity to thestationary processing system in order to use the interface.

What is needed is a completely wearable (i.e., completely portable) userinterface that enables a user to see, access and interact with anelectronic display in an inconspicuous, hands-free manner.

BRIEF SUMMARY

The present disclosure relates to a muscle interface device and methodfor interacting with content displayed on wearable head mounteddisplays.

More generally, the muscle interface device comprises a sensor worn onthe forearm of a user, and the sensor is adapted to recognize aplurality of gestures made by a user's hand and or wrist to interactwith content displayed on the wearable head mounted display.

In an embodiment, the muscle interface device utilizes a plurality ofelectromyographic (EMG) sensors to detect electrical activity producedby muscles during contraction, and convert the electrical signals forprocessing. The electrical signals detected from the muscles areinterpreted as gestures (e.g. a combination of hand, wrist and armmovements) made by a user which provide a control input to a wearablehead mounted display. The control input is preferably providedwirelessly via a wireless communication protocol, such as Near-FieldCommunication (“NFC”) or Bluetooth™, for example.

In another embodiment, various types of sensors may be used alone or inlieu of or in combination with EMG sensors to detect gestures made by auser, for processing as a control input for interacting with a wearablehead mounted display. This may be one or more mechanomyographic (MMG)sensors to detect vibrations made by muscles during contraction, or oneor more accelerometer sensors to detect larger movements.

In another embodiment, the muscle interface device includes acalibration module with a routine for calibrating the muscle interfacedevice for use with the wearable head mounted display.

Other features and advantages will become apparent from the followingdetailed description and accompanying drawings. It should be understood,however, that the detailed description and specific examples are givenby way of illustration and not limitation. Many modifications andchanges within the scope of the present invention may be made withoutdeparting from the spirit thereof, and the invention includes all suchmodifications.

A wearable muscle interface device that in use interacts with contentdisplayed on a wearable head-mounted display may be summarized asincluding: a plurality of muscle activity sensors to be worn on an armof a user, the muscle activity sensors responsive to signals generatedby muscles in the arm of the user; and a transmitter communicativelycoupled to the plurality of muscle activity sensors, wherein in use thetransmitter transmits at least one signal from the wearable muscleinterface device directly to a receiver on the wearable head-mounteddisplay based on the signals detected by the muscle activity sensors;wherein the at least one signal transmitted, in use, from the wearablemuscle interface device directly to the receiver on the wearablehead-mounted display effects at least one interaction with contentdisplayed on the wearable head-mounted display. The wearable muscleinterface device may further include a processor that in use interpretsthe signals detected by the muscle activity sensors as a gesture,wherein the processor is communicatively coupled in between thetransmitter and the plurality of muscle activity sensors, and whereinthe at least one signal that, in use, is transmitted from the wearablemuscle interface device may be based on the gesture interpreted by theprocessor of the wearable muscle interface device. The wearablehead-mounted display may include a processor communicatively coupled tothe receiver of the wearable head-mounted display, and the at least onesignal that, in use, is transmitted from the wearable muscle interfacedevice to the wearable head-mounted display may be interpreted as agesture by the processor of the wearable head-mounted display.

The wearable muscle interface device may further include a hapticfeedback module that in use provides haptic feedback to the user, thehaptic feedback module including a vibratory motor. The plurality ofmuscle activity sensors may include at least one muscle activity sensorselected from the group consisting of: an electromyographic (EMG) sensorand a mechanomyographic (MMG) sensor. The wearable muscle interfacedevice may further include at least one accelerometer that in usedetects signals generated by motion of the arm of the user, the at leastone accelerometer communicatively coupled to the transmitter, andwherein in use the at least one signal transmitted from the transmitterof the wearable muscle interface device directly to the receiver on thewearable head-mounted display may be based on both the signals detectedby the muscle activity sensors and the signals detected by the at leastone accelerometer. The transmitter may include a wireless transmitter.

A wearable system that in use provides hands-free access to and controlof a portable electronic display may be summarized as including: i) awearable muscle interface device comprising: a plurality of muscleactivity sensors to be worn on an arm of a user, the muscle activitysensors responsive to signals generated by muscles in the arm of theuser; and a transmitter communicatively coupled to the plurality ofmuscle activity sensors, wherein in use the transmitter transmits atleast one signal from the wearable muscle interface device based on thesignals detected by the muscle activity sensors; and ii) a wearablehead-mounted display comprising: at least one display screen to be wornon a head of the user, the at least one display screen arranged to bepositioned in front of at least one eye of the user when worn on thehead of the user; a receiver communicatively coupled to the at least onedisplay screen, wherein in use the receiver directly receives the atleast one signal transmitted from the transmitter of the wearable muscleinterface device; and a processor communicatively coupled to thereceiver and to the at least one display screen, wherein in use the atleast one signal received directly from the transmitter of the wearablemuscle interface device by the receiver of the wearable head-mounteddisplay effects control of at least one function of the wearablehead-mounted display. The transmitter of the wearable muscle interfacedevice may include a wireless transmitter and the receiver of thewearable head-mounted display may include a wireless receiver. Thewearable muscle interface device of the wearable system may furtherinclude a processor that in use interprets the signals detected by themuscle activity sensors as a gesture, wherein the processor of thewearable muscle interface device is communicatively coupled in betweenthe transmitter and the plurality of muscle activity sensors, andwherein the at least one signal that, in use, is transmitted from thewearable muscle interface device may be based on the gesture interpretedby the processor of the wearable muscle interface device.

The plurality of muscle activity sensors in the wearable muscleinterface device of the wearable system may include at least one muscleactivity sensor selected from the group consisting of: anelectromyographic (EMG) sensor and a mechanomyographic (MMG) sensor. Thewearable muscle interface device of the wearable system may furtherinclude at least one accelerometer that in use detects signals generatedby motion of the arm of the user, the at least one accelerometercommunicatively coupled to the transmitter, and wherein in use the atleast one signal transmitted by the transmitter of the wearable muscleinterface device may be based on both the signals detected by the muscleactivity sensors and the signals detected by the at least oneaccelerometer.

A method of using a wearable system to achieve hands-free access to andcontrol of a portable electronic display, wherein the wearable systemincludes a wearable muscle interface device and a wearable head-mounteddisplay, may be summarized as including: detecting muscle activitycorresponding to a physical gesture performed by a user of the wearablesystem by at least one muscle activity sensor of the wearable muscleinterface device; transmitting at least one signal from the wearablemuscle interface device by a transmitter of the wearable muscleinterface device based at least in part on the muscle activity detectedby at least one muscle activity sensor of the wearable muscle interfacedevice; receiving the at least one signal directly from the wearablemuscle interface device by a receiver of the wearable head-mounteddisplay; processing the at least one signal by a processor of thewearable head-mounted display; and effecting at least one interactionbetween the user and the wearable head-mounted display by the processorof the wearable head-mounted display based on the processing of the atleast one signal by the processor of the wearable head-mounted display.The method may further include, in response to detecting muscle activitycorresponding to a physical gesture performed by a user of the wearablesystem by at least one muscle activity sensor of the wearable muscleinterface device, processing the detected muscle activity by a processorof the wearable muscle interface device, and transmitting at least onesignal from the wearable muscle interface device by a transmitter of thewearable muscle interface device based at least in part on the muscleactivity detected by at least one muscle activity sensor of the wearablemuscle interface device may include transmitting at least one signalfrom the wearable muscle interface device by the transmitter of thewearable muscle interface device based at least in part on processingthe detected muscle activity by the processor of the wearable muscleinterface device.

The method may further include detecting motion of the wearable muscleinterface device corresponding to the physical gesture performed by theuser of the wearable system by at least one accelerometer of thewearable muscle interface device, and transmitting at least one signalfrom the wearable muscle interface device by a transmitter of thewearable muscle interface device based at least in part on the muscleactivity detected by at least one muscle activity sensor of the wearablemuscle interface device may include transmitting at least one signalfrom the wearable muscle interface device by the transmitter of thewearable muscle interface device based on both the muscle activitydetected by at least one muscle activity sensor of the wearable muscleinterface device and the motion detected by at least one accelerometerof the wearable muscle interface device. Transmitting at least onesignal from the wearable muscle interface device by a transmitter of thewearable muscle interface device may include wirelessly transmitting atleast one signal from the wearable muscle interface device by a wirelesstransmitter of the wearable muscle interface device. Receiving the atleast one signal directly from the wearable muscle interface device by areceiver of the wearable head-mounted display may include wirelesslyreceiving the at least one signal directly from the wearable muscleinterface device by a wireless receiver of the wearable head-mounteddisplay.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

In the drawings, identical reference numbers identify similar elementsor acts. The sizes and relative positions of elements in the drawingsare not necessarily drawn to scale. For example, the shapes of variouselements and angles are not drawn to scale, and some of these elementsare arbitrarily enlarged and positioned to improve drawing legibility.Further, the particular shapes of the elements as drawn are not intendedto convey any information regarding the actual shape of the particularelements, and have been solely selected for ease of recognition in thedrawings.

FIG. 1 is a side plan view that illustrates a user wearing a headmounted display and a muscle interface device in accordance with thepresent systems, devices, and methods.

FIG. 2A is an isometric view that illustrates a detailed view of amuscle interface device in accordance with the present systems, devices,and methods.

FIG. 2B is a data graph that illustrates an electrical signal detectedby an EMG sensor.

FIG. 3 is a schematic view that illustrates wireless communicationbetween a head mounted display and a muscle interface device inaccordance with the present systems, devices, and methods.

FIG. 4 is a schematic view that illustrates a user's hand and wristgesture processed as a control signal by the muscle interface device forinteracting with content displayed on the head mounted display.

FIG. 5 is a schematic view of a system architecture of a muscleinterface device in accordance with the present systems, devices, andmethods.

FIG. 6 is a flow chart of a method of using a wearable system to achievehands-free access to and control of a portable electronic display inaccordance with the present systems, devices, and methods.

FIG. 7 is a flow-diagram showing a method of using wearable system toachieve hands-free access to and control of a portable electronicdisplay in accordance with the present systems, devices, and methods.

In the drawings, embodiments of the invention are illustrated by way ofexample. It is to be expressly understood that the description anddrawings are only for the purpose of illustration and as an aid tounderstanding, and are not intended as a definition of the limits of theinvention.

DETAILED DESCRIPTION

In the following description, certain specific details are set forth inorder to provide a thorough understanding of various disclosedembodiments. However, one skilled in the relevant art will recognizethat embodiments may be practiced without one or more of these specificdetails, or with other methods, components, materials, etc. In otherinstances, well-known structures associated with electronic devices, andin particular portable electronic devices such as wearable electronicdevices, have not been shown or described in detail to avoidunnecessarily obscuring descriptions of the embodiments.

Unless the context requires otherwise, throughout the specification andclaims which follow, the word “comprise” and variations thereof, suchas, “comprises” and “comprising” are to be construed in an open,inclusive sense, that is as “including, but not limited to.”

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structures, orcharacteristics may be combined in any suitable manner in one or moreembodiments.

As used in this specification and the appended claims, the singularforms “a,” “an,” and “the” include plural referents unless the contentclearly dictates otherwise. It should also be noted that the term “or”is generally employed in its broadest sense, that is as meaning “and/or”unless the content clearly dictates otherwise.

The headings and Abstract of the Disclosure provided herein are forconvenience only and do not interpret the scope or meaning of theembodiments.

The present disclosure relates to muscle interface systems, devices andmethods that enable a user to access and interact with content displayedon an electronic display in an inconspicuous, hands-free manner.

In an aspect, a wearable system includes a wearable muscle interfacedevice comprising a plurality of muscle activity sensors worn on an armof a user. The plurality of muscle activity sensors are responsive tosignals generated by muscles in the arm of the user. For example, whenthe user performs a physical gesture that involves one or more muscle(s)in the arm upon which the muscle interface device is worn, at least oneof the muscle activity sensors may detect signals generated by the oneor more muscle(s). The wearable muscle interface device is adapted torecognize gestures made by the user and to interact with contentdisplayed on a wearable head-mounted display in response to therecognized gestures. To this end, the wearable system further includes awearable head-mounted display and the wearable muscle interface deviceincludes a transmitter communicatively coupled to the plurality ofmuscle activity sensors. In use, the transmitter of the wearable muscleinterface device transmits at least one signal from the wearable muscleinterface device directly to a receiver on the wearable head-mounteddisplay based on the signals detected by the muscle activity sensors.The at least one signal transmitted from the wearable muscle interfacedevice directly to the receiver on the wearable head-mounted displayeffects at least one interaction with content displayed on the wearablehead-mounted display.

In another aspect, a muscle interface method comprises processing atleast one signal based on one or more gesture(s) made by a user's hand,wrist and/or arm movements to interact with content displayed on thewearable head-mounted display.

The plurality of muscle activity sensors in and/or on-board the wearablemuscle interface device may include electromyography (EMG) sensorsand/or mechanomyography (MMG) sensors to detect electrical signalsand/or vibrations, respectively, produced by muscles in the user's armand to provide one or more signal(s) in response to the detectedelectrical signals and/or vibrations. The electrical signals and/orvibrations detected from the muscles are interpreted as gestures made bythe user which provide a direct control input to a wearable head-mounteddisplay.

The control input is provided directly from the wearable muscleinterface device to the wearable head-mounted display. Preferably, thecontrol input is provided wirelessly from the wearable muscle interfacedevice directly to the wearable head-mounted display via a wirelesscommunication protocol, such as NFC or Bluetooth™, for example. However,it will be appreciated that other types of wireless communications maybe used, including any wireless communication protocol developed forsmart phones and similar devices. In some applications, a direct wireconnection between the wearable muscle interface device and the wearablehead-mounted display may be used.

In addition to EMG and/or MMG sensors, various other types of sensorsmay be used to detect gestures made by the user. For example, inertialsensors such as accelerometers and/or gyroscopes may be used to detectsignals generated by motion of the arm of the user in response to theuser performing the physical gesture. The wearable muscle interfacedevice may include one or more accelerometer sensors that, in use,detect signals generated by motion of the arm of the user and/or measurecharacteristics of gestures made by the user, including gesturesinvolving the elbow or even the shoulders of the user. When usedtogether with EMG and/or MMG sensors for detecting gestures, theaccelerometer sensors may be utilized to increase the variety of controlinputs that may be generated for direct interaction with a wearablehead-mounted display.

An illustrative example will now be described with reference to thedrawings.

Shown in FIG. 1 is an illustrative user 100 wearing a wearable system150 that in use provides hands-free access to and control of a portableelectronic display 310 in accordance with the present systems, devices,and methods. Wearable system 150 includes a wearable head-mounteddisplay 310 with on-board display control 300, and a wearable muscleinterface device 200 having a plurality of muscle activity sensors inaccordance with the present systems, devices, and methods. In thisillustrative example, wearable muscle interface device 200 is aflexible, stretchable band that may be worn on the arm (e.g., theforearm) of user 100 as shown. As discussed in more detail herein,wearable muscle interface device 200 includes a transmitter (e.g., awireless transmitter) and wearable head-mounted display 310 includes areceiver (e.g., a wireless receiver) such that at least one signal maybe transmitted from wearable muscle interface device 200 directly towearable head-mounted display 310 (i.e., without being received andre-transmitted by any intervening device, such as a stationary,non-portable intervening device) in response to signals detected by themuscle activity sensors of wearable muscle interface device 200 in orderto effect interactions with and/or control of content displayed on or bywearable head-mounted display 310.

FIG. 2A illustrates a detailed view of wearable muscle interface device200 from wearable system 150 of FIG. 1 in accordance with the presentsystems, devices, and methods. As shown, wearable muscle interfacedevice 200 may comprise a processor 210 (e.g., a central processingunit, a digital microcontroller, a digital signal processor, orsimilar), and one or more batteries 220, which may be rechargeable, andwhich may be utilized concurrently or sequentially in conventionalmanner. As shown, wearable muscle interface device 200 is a band to beworn on an arm of a user (e.g., a forearm of a user) and includes aplurality of muscle activity sensors 230 which may be positionedradially around the circumference of the band, such that the sensors 230can, when in use, detect signals generated by muscles in the arm of user100 in response to user 100 performing a physical gesture. Wearablemuscle interface device 200 may further include transmitter 250 (e.g., awireless transmitter) communicatively coupled to the plurality of muscleactivity sensors 230 which, in use, transmits at least one signal fromwearable muscle interface device 200 directly to a receiver on awearable head-mounted display 310 based on the signals detected bymuscle activity sensors 230. Wearable muscle interface device 200 mayinclude a feedback mechanism (e.g., a haptic feedback module) such as avibratory motor 240 to provide haptic feedback as described furtherbelow.

Further exemplary details that may be included in wearable muscleinterface device 200 include the systems, articles, and methodsdescribed in, without limitation: U.S. Provisional Patent ApplicationSer. No. 61/768,322, U.S. Provisional Patent Application Ser. No.61/771,500, U.S. Provisional Patent Application Ser. No. 61/857,105,U.S. Provisional Patent Application Ser. No. 61/860,063, U.S.Provisional Patent Application Ser. No. 61/822,740, U.S. ProvisionalPatent Application Ser. No. 61/866,960, U.S. Provisional PatentApplication Ser. No. 61/869,526, U.S. Provisional Patent ApplicationSer. No. 61/874,846, U.S. Provisional Patent Application Ser. No.61/872,569, U.S. Provisional Patent Application Ser. No. 61/881,064,U.S. Provisional Patent Application Ser. No. 61/894,263, U.S.Provisional Patent Application Ser. No. 61/903,238, U.S. ProvisionalPatent Application Ser. No. 61/909,786, and/or U.S. Provisional PatentApplication Ser. No. 61/915,338, all of which are incorporated byreference herein in their entirety.

Wearable muscle interface device 200 may be calibrated when first worn,prior to operation, such that muscle interface device 200 may performreliable gesture identification regardless of the exact positioning ofthe muscle activity sensors 230 on the user's arm.

By way of example, muscle activity sensors 230 may include one or moreEMG sensor(s), each of which may provide a respective EMG signal in theform of an oscillating waveform that varies in both frequency andamplitude. A majority of signal information that is needed for reliablegesture identification may be contained within a limited bandwidth ofsuch an oscillating waveform, such as in the 5 Hz to 250 Hz frequencyband. An illustrative example of an EMG signal 200B is shown in FIG. 2B.

As previously described, the plurality of muscle activity sensors 230may include one or more MMG sensor(s) comprising piezoelectric sensors,which may be used to measure the vibrations at the surface of the skinproduced by the underlying muscles when contracted. By way of example,the MMG signal generated may be an oscillating waveform that varies inboth frequency and amplitude, and a majority of signal information thatis needed for reliable gesture identification may be contained within alimited bandwidth, such as in the 5 Hz to 250 Hz frequency band. Becausethe MMG signal is acquired via mechanical means, electrical variationslike skin impedance may not have a significant effect on the signal. TheMMG signal may be very similar to the illustrative example of EMG signal200B shown in FIG. 2B.

As previously described, wearable muscle interface device 200 mayinclude one or more accelerometer sensor(s) 260 that, in use, detectadditional aspects of gestures made by user 100 in, for example, threedegrees of freedom. For example, at least one accelerometer 260 may becommunicatively coupled to transmitter 250 of wearable muscle interfacedevice 200 and, in use, the at least one signal transmitted fromtransmitter 250 directly to the receiver on the wearable head-mounteddisplay 310 may be based on both the signals detected by muscle activitysensors 230 and the signals detected by the at least one accelerometer260. An accelerometer signal may, for example, consist of three digitalchannels of data, each representing the acceleration in a respective oneof three orthogonal directions (e.g., the x, y, and z directions). Thesignal may be representative of all of the accelerations that the user'sarm is subject to, and may further represent motion of the body as awhole.

Now referring to FIG. 3, shown is wearable system 150 from FIG. 1 withan illustration of direct wireless communication (e.g., Bluetooth™, NFC,etc.) between wearable muscle interface device 200 and wearablehead-mounted display 310 in accordance with the present systems,devices, and methods. This wireless communication is utilized totransmit one or more signal(s) from wearable muscle interface device 200directly to wearable head-mounted display 310 (e.g., to a wirelessreceiver 350 located in or on display control 300 of wearablehead-mounted display 310) without any intervening communicativecouplings or links. In this way, the user 100 may access and control orotherwise interact with a portable electronic display in aninconspicuous and hands-free manner. User 100 does not need to use hisor her hand(s) to position or orient the portable electronic display ofwearable head-mounted display 310 in order to be able to see, access,receive feedback from, or otherwise interact with the portableelectronic display of wearable head-mounted display 310 because wearablehead-mounted display 310 is arranged such that at least one displayscreen is positioned in front of at least one eye of user 100 at alltimes while wearable head-mounted display 310 is worn on user 100'shead, regardless of the direction that user 100 is facing. Furthermore,wearable muscle interface device 200 enables user 100 to control orotherwise interact with content displayed on wearable head-mounteddisplay 310 in an inconspicuous manner by using touchless gestures.

Inconspicuous gesture-based control of and/or interactions with wearablehead-mounted display 310 is illustrated by way of example in FIG. 4, inwhich user 100's hand and wrist gesture is detected and processed bywearable muscle interface device 200 and transmitted directly fromtransmitter 250 to receiver 350 of wearable head-mounted display 310 forinteracting with content displayed thereon.

In this particular example, a gesture 410 made by the user (100)extending an index finger, and making a wrist flexion motion 420 isdetected by the muscle activity sensors 230 (and/or accelerometersensors 260 if included) of wearable muscle interface device 200 (notvisible in FIG. 4). Signals provided by the muscle activity sensors 230in response to the detected gesture 410 are processed by processor 210(FIG. 2) which interprets the signals to identify gesture 410 performedby user 100. A corresponding signal is produced based on the gesture 410interpreted by the processor 210 and the signal is transmitted fromtransmitter 250 directly to receiver 350 of wearable head-mounteddisplay 310, which causes a menu appearing on wearable head-mounteddisplay 310 to scroll downwards.

As another example, a similar gesture in which user 100 extends theindex finger and makes a wrist extension motion may be detected bymuscle activity sensors 230 (and/or accelerometer sensors 260 ifincluded) of wearable muscle interface device 200 and processed byprocessor 210 (FIG. 2). Processor 210 may interpret the detected muscleactivity to identify the gesture performed, and a corresponding signalmay be transmitted from transmitter 250 directly to receiver 350 ofwearable head-mounted display 310 to cause a menu appearing on wearablehead-mounted display 310 to scroll upwards.

As yet another example, a gesture in which user 100 extends the indexfinger and makes a poking motion involving a slight movement of theelbow and shoulder may be detected by muscle activity sensors 230(and/or accelerometer sensors 260 if included) of wearable muscleinterface device 200 and processed by processor 210 (FIG. 2). Processor210 may interpret the detected muscle activity to identify the gestureperformed, and a corresponding signal may be transmitted fromtransmitter 250 directly to receiver 350 of wearable head-mounteddisplay 310 to cause a highlighted menu item appearing on wearablehead-mounted display 310 to be selected.

If the user extends a different finger other than the index finger,muscle activity sensors 230 may detect this, a different gesture may beidentified by wearable muscle interface device 200, and a differentsignal may be transmitted directly to wearable head-mounted display 310to effect a different interaction or function thereof. For example,extending the little finger or “pinky” finger instead of the indexfinger may cause wearable system 150 to interpret the user's gestureswith functions analogous to clicking a right mouse button rather than aleft mouse button in a conventional mouse user interface. Extending boththe index and pinky fingers at the same time may cause wearable system150 to interpret the user's gestures with yet other functions analogousto clicking a third mouse button in a conventional mouse user interface.

Thus, wearable muscle interface device 200 may be adapted and/orcalibrated to recognize a wide range of gestures made by a user 100,based on measurements from a plurality of muscle activity sensors 230(and, in some implementations, one or more accelerometer sensor(s) 260)in the wearable muscle interface device 200.

Wearable muscle interface device 200 may itself be operative tointerpret the gestures from the detected signals as described above by,for example, using an on-board processor 210 to process the EMG signalsand interpret the EMG signals as a gesture via a gesture identificationprocess (e.g., by invoking data and/or instructions stored in anon-board non-transitory computer-readable storage medium that, whenexecuted by processor 210, cause processor 210 to identify the gestureperformed by user 100). Wearable muscle interface device 200 may thentransmit one or more signal(s) from transmitter 250 directly to receiver350 of wearable head-mounted display 310 in order to effect someinteraction with wearable head-mounted display 310 based on theinterpreted gesture. In this example, the processor 210 may becommunicatively coupled in between the transmitter 250 and the pluralityof muscle activity sensors 230 such that transmitter 250 transmits oneor more signal(s) provided by processor 210 (e.g., corresponding to aninterpreted gesture) based at least in part on the signals provided bymuscle activity sensors 230.

However, in an alternative implementation, the detected EMG signals maybe transmitted directly to the receiver 350 of wearable head-mounteddisplay 310 from transmitter 250 (e.g., without being processed byprocessor 210, which may or may not be included in device 200 in thisexample) and wearable head-mounted display 310 may include a processor320 (e.g., a central processing unit, a digital microcontroller, adigital signal processor, or similar, located in or on display control300) communicatively coupled to receiver 350 to process the EMG signalsand interpret the EMG signals as a gesture via a gesture identificationprocess (e.g., by invoking data and/or instructions stored in anon-board non-transitory computer-readable storage medium that, whenexecuted by processor 320, cause processor 320 to identify the gestureperformed by user 100). Wearable head-mounted display 310 may theneffect some interaction with content displayed thereon based on theinterpreted gesture. Whether the detected EMG signals are interpreted atthe device 200 or at the display 310, the detected EMG signals are firstinterpreted as a recognized gesture in order to interact with contentdisplayed on the display 310.

Wearable muscle interface device 200 may include a haptic feedbackmodule to provide feedback that a gesture has been recognized. Thishaptic feedback may provide a user 100 with confirmation that the user100's gesture has been recognized, and successfully converted to asignal to interact with content displayed on wearable head-mounteddisplay 310. The haptic feedback module may comprise, for example, avibrating mechanism such as a vibratory motor 240 built into thewearable muscle interface device 200.

Alternatively, rather than haptic feedback provided by the wearablemuscle interface device 200, confirmation of recognition of a gesturemay be provided by auditory feedback, either generated by a speaker onthe wearable muscle interface device 200, or operatively connected tothe wearable head-mounted display 310.

As another alternative, confirmation of recognition of a gesture may beprovided visually on the wearable head-mounted display 310 itself. Ifthere is more than one possible gesture that may be interpreted from thedetected signals, rather than providing a possibly erroneous signal, thewearable muscle interface device 200 and/or the wearable head-mounteddisplay 310 may provide a selection of two or more possible gestures aspossible interpretations, and the user may be prompted to select fromone of them to confirm the intended gesture and corresponding control.

Now referring to FIG. 5, shown is an illustrative schematic systemarchitecture 500 of the wearable muscle interface device 200 componentof a wearable system 150 providing inconspicuous and hands-free accessto and control of a portable electronic display in accordance with thepresent systems, device, and methods. As shown, system architecture 500includes a CPU 502 (e.g., a processor, such as a digital microprocessoror microcontroller), non-transitory computer-readable memory 504, systemclock 506, a wireless communication module 508 (e.g., Bluetooth™, NFC,or the like), and a direct memory access (DMA) controller 510. As shown,DMA controller 510 is adapted to receive inputs from various sensorson-board the wearable muscle interface device 200, including one or moreEMG sensors 520, MMG sensors 530 and/or accelerometer sensors 540.

In the illustrative example of system architecture 500, detected signalsfrom one or more EMG sensors 520 are processed through signal filter 522and converted from analog to digital signals by ADC 524. If one or moreMMG sensors 530 are used (either in addition to or instead of EMGsensors 520), then the detected signals from the MMG sensors 530 areprocessed through signal filter 532 and converted from analog to digitalsignals by ADC 534. Digital signals from one or more accelerometersensors 540 may also be processed through signal filter 542 and receivedby DMA controller 510.

The data from the various types of sensors 520, 530, 540 may be acquiredthrough an analog filtering chain. The data may be band-passed throughfilters 522, 532 between about 10 Hz to about 500 Hz, and amplified(e.g. by a total of about 1000 times). This filtering and amplificationcan be altered to whatever is required to be within software parameters.A notch filter at 60 Hz, or at any other relevant frequency, may also beused to remove powerline noise.

Data from the sensors 520, 530 may be converted to, e.g., 12-bit digitaldata by ADCs 524, 534, and then clocked into onboard memory 504 usingclock 506 by the DMA controller 510 to be processed by the CPU 502.

Now referring to FIG. 6, shown is a schematic flow chart of a method 600of using a wearable system (e.g., 150) to achieve hands-free access toand control of a portable electronic display in accordance with thepresent systems, devices, and methods. As shown, method 600 begins atblock 602, where method 600 pairs a wearable muscle interface device 200with a wearable head-mounted display 310. Method 600 then proceeds toblock 604, where content and/or user interface (UI) is displayed on thewearable head-mounted display 310.

Method 600 then proceeds to block 606, where method 600 determines ifthe displayed content and/or UI is navigable. If no, method 600 returnsto block 604. If yes, method 600 proceeds to block 608, where thewearable muscle interface device 200 detects muscle activitycorresponding to a physical gesture performed by a user of the wearablesystem 150 (i.e., at least one muscle activity sensor 230 of thewearable muscle interface device 200 detects the user's intentionalhand/arm movements and positions), and wirelessly sends/transmits atleast one signal corresponding to an identified gesture from thewearable muscle interface device 200 to the wearable head-mounteddisplay 310. The at least one signal may be sent by a transmitter 250 ofthe wearable muscle interface device 200 based on the muscle activitydetected by at least one muscle activity sensor 230 of the wearablemuscle interface device 200.

Method 600 then proceeds to block 610, where a receiver 350 on thewearable head-mounted display 310 receives the at least one signaldirectly from the transmitter 250 of the wearable muscle interfacedevice 200. A processor 320 of the wearable head-mounted display 310processes the at least one signal, and effects at least one interactionbetween the user 100 and the wearable head-mounted display 310 based onthe processing of the at least one signal by processor 320 of thewearable head-mounted display 310.

Another example of a method employing a wearable system in accordancewith the present systems, devices, and methods is illustrated in FIG. 7.FIG. 7 is a flow-diagram showing a method 700 of using wearable system150 to achieve hands-free access to and control of a portable electronicdisplay. The wearable system 150 includes a wearable muscle interfacedevice 200 and a wearable head-mounted display 310. Method 700 includesfive acts 701, 702, 703, 704, and 705, although those of skill in theart will appreciate that in alternative embodiments certain acts may beomitted and/or additional acts may be added. Those of skill in the artwill also appreciate that the illustrated order of the acts is shown forexemplary purposes only and may change in alternative embodiments. Forthe purpose of method 700, the term “user” refers to a person that iswearing both the wearable muscle interface device 200 (e.g., worn on atleast one of the user's arms) and the wearable head-mounted display 310of the wearable system 150 (e.g., worn on the user's head).

At 701, the user performs a physical gesture and muscle activitycorresponding to the physical gesture is detected by at least muscleactivity sensor 230 of the wearable interface device 200. The muscleactivity sensors 230 may include at least one EMG sensor that detectselectrical signals generated by the muscle activity and/or at least oneMMG sensor that detects vibrations generated by the muscle activity. Inaddition to muscle activity, motion of the wearable muscle interfacedevice 200 corresponding to the physical gesture may be detected by atleast one accelerometer 260 on-board the wearable muscle interfacedevice 200.

At 702, at least one signal is transmitted by a transmitter 250 of thewearable muscle interface device 200 based at least in part on themuscle activity detected at 701. As previously described, transmitter250 may be a wireless transmitter such that transmitting at least onesignal by transmitter 250 includes wirelessly transmitting the at leastone signal by transmitter 250. In implementations in which motion of thewearable muscle interface device 200 is also detected by at least oneaccelerometer 260, transmitting at least one signal by transmitter 250based at least in part on the muscle activity detected at 701 mayinclude transmitting at least one signal by transmitter 250 based onboth the muscle activity detected by at least one muscle activity sensor230 and the motion detected by at least one accelerometer 260.

In response to detecting muscle activity corresponding to a physicalgesture performed by the user at 701, method 700 may include processingthe detected muscle activity by a processor 210 communicatively coupledin between the muscle activity sensors 230 and the transmitter 250(e.g., to interpret the signals provided by the muscle activity sensors230 and/or to identify the user-performed gesture). In this case,transmitting at least one signal by transmitter 250 based at least inpart on the muscle activity detected at 701 may include transmitting atleast one signal by transmitter 250 based at least in part on processingthe detected muscle activity by the processor 210 of the wearable muscleinterface device 200.

At 703, the at least one signal is received directly from transmitter250 by a receiver 350 of the wearable head-mounted display 310. Inimplementations where transmitter 250 is a wireless transmitter,receiver 350 may include a wireless receiver such that receiving the atleast one signal by receiver 350 includes wirelessly receiving the atleast one signal by receiver 350. The at least one signal is transmitteddirectly from transmitter 250 to receiver 350 without routing throughany intervening devices or systems.

At 704, the at least one signal received by receiver 350 is processed bya processor 320 of the wearable head-mounted display 320. Processing theat least one signal by the processor 320 of the wearable head-mounteddisplay may include, for example, mapping or otherwise associating theat least one signal to/with one or more function(s) of the wearablehead-mounted display 310 based on data and/or instructions stored in anon-transitory computer-readable storage medium on-board the wearablehead-mounted display 310 (data and/or instructions which, when executedby the processor 320 of the wearable head-mounted display 310, cause theprocessor 320 of the wearable head-mounted display to effect one or morefunction(s) of the wearable head-mounted display 310).

At 705, at least one interaction between the user and the wearablehead-mounted display 310 is effected by the processor 320 of thewearable head-mounted display 310 based on the processing of the atleast one signal at 704. The at least one interaction may include anyfunction or operation that prompts, modifies, changes, elicits, orotherwise involves visual information provided to the user by thewearable head-mounted display 310, including without limitation:interacting with visual material such as a photograph or video,navigating a menu, interacting with visually displayed elements such asa map or an element of a video game, and so on. Depending on thespecific application, elements displayed on the wearable head-mounteddisplay 310 may or may not accommodate or otherwise take into accountaspects of the user's environment that may be visible to the user. Forexample, elements displayed on the wearable head-mounted display 310 mayobscure, overlay, augment, highlight, block, be superimposed on, and/orsemi-transparently project in front of elements of the user'senvironment.

As will be appreciated, the systems, devices, and methods that enable auser to access and interact with content displayed on an electronicdisplay in an inconspicuous, hands-free manner described herein may beused for interaction with a portable electronic display in a wide rangeof applications, in virtually any application in which portableelectronic displays are contemplated. By providing a discreet method ofinteracting with a wearable head-mounted display, a user is able tointeract with such a display in any operating environment, includingsituations where overt gesturing (e.g. raising the hand to touch aninput device provided on the wearable head-mounted display itself) isnot desirable.

While various embodiments and illustrative examples have been describedabove, it will be appreciated that these embodiments and illustrativeexamples are not limiting, and the scope of the invention is defined bythe following claims.

The various embodiments described herein provide, at least, a wearablesystem (e.g., 150) including a wearable muscle interface device (e.g.,200) that, in use, is to be worn on an arm of a user in order to enablehands-free access to, and control of, a wearable head-mounted display(e.g., 310). As described previously, the singular forms “a,” “an,” and“the” used in this specification and the appended claims include pluralreferents unless the content clearly dictates otherwise. In someapplications, it can be advantageous or otherwise desirable for such awearable system (150) to employ two or more wearable muscle interfacedevices (e.g., two or more wearable muscle interface devices 200) wornon both of the user's arms (e.g., at least a respective wearable muscleinterface device 200 worn on each of the user's arms) as described inU.S. Provisional Patent Application Ser. No. 61/874,846. Such may enablea greater number and/or diversity of gestures to be used to interactwith content displayed on the wearable head-mounted display (e.g., 310).Furthermore, in various embodiments the gesture-based interactionsystems, devices, and methods described herein may be combined withother forms of touchless control, including without limitation:voice/speech-based control techniques such as Siri®, control techniquesbased on eye/vision tracking and/or blinking, electroencephalography(EEG), or the like.

Throughout this specification and the appended claims, the terms“head-mounted display” and “heads-up display” are used substantiallyinterchangeably to refer to an electronic display that is worn on thehead of a user and arranged so that at least one electronic display ispositioned in front of at least one eye of the user when thehead-mounted/heads-up display is worn on the head of the user. Forgreater clarity, “positioned in front of at least one eye of the user”means that the content displayed on or by the electronic display isdisplayed, projected, or otherwise provided generally in front of atleast one eye of the user and is visible by that at least one eyeregardless of the orientation or position of the user's head. Anelectronic display that is “positioned in front of at least one eye ofthe user” may correspond to a projection, reflection, refraction,diffraction, or direct display of optical signals and may be located inthe user's direct line of sight or may be located off of the user'sdirect line of sight such that the user may or may not need todeliberately direct one or more eye(s), without necessarily moving theirhead, towards the electronic display in order to see (i.e., access) thecontent displayed thereby.

Throughout this specification and the appended claims, the term“gesture” is used to generally refer to a physical action (e.g., amovement, a stretch, a flex, a pose) performed or otherwise effected bya user. Any physical action performed or otherwise effected by a userthat involves detectable muscle activity (detectable, e.g., by at leastone appropriately positioned muscle activity sensor) and/or detectablemotion (detectable, e.g., by at least one appropriately positionedinertial sensor, such as an accelerometer and/or a gyroscope) mayconstitute a gesture in the present systems, articles, and methods.

Throughout this specification and the appended claims the term“communicative” as in “communicative pathway,” “communicative coupling,”and in variants such as “communicatively coupled,” is generally used torefer to any arrangement for transferring and/or exchanging information.Exemplary communicative pathways include, but are not limited to,electrically conductive pathways (e.g., electrically conductive wires,electrically conductive traces), magnetic pathways (e.g., magneticmedia), and/or optical pathways (e.g., optical fiber), and exemplarycommunicative couplings include, but are not limited to, electricalcouplings, magnetic couplings, and/or optical couplings.

Throughout this specification and the appended claims, the term“provide” and variants such as “provided” and “providing” are frequentlyused in the context of signals. For example, a muscle activity sensor isdescribed as “providing at least one signal” and an inertial sensor isdescribed as “providing at least one signal.” Unless the specificcontext requires otherwise, the term “provide” is used in a most generalsense to cover any form of providing a signal, including but not limitedto: relaying a signal, outputting a signal, generating a signal, routinga signal, creating a signal, transducing a signal, and so on. Forexample, a surface EMG sensor may include at least one electrode thatresistively or capacitively couples to electrical signals from muscleactivity. This coupling induces a change in a charge or electricalpotential of the at least one electrode which is then relayed throughthe sensor circuitry and output, or “provided,” by the sensor. Thus, thesurface EMG sensor may “provide” an electrical signal by relaying anelectrical signal from a muscle (or muscles) to an output (or outputs).In contrast, an inertial sensor may include components (e.g.,piezoelectric, piezoresistive, capacitive, etc.) that are used toconvert physical motion into electrical signals. The inertial sensor may“provide” an electrical signal by detecting motion and generating anelectrical signal in response to the motion.

Throughout this specification and the appended claims, “identifying” or“interpreting signals as” a gesture means associating a set of signalsprovided by one or more muscle activity sensor(s) (230) with aparticular gesture. In the various embodiments described herein,“identifying” or “interpreting signals as” a gesture includesdetermining which gesture in a gesture library is most probable(relative to the other gestures in the gesture library) of being thegesture that a user has performed or is performing in order to producethe signals upon which the gesture identification is at least partiallybased. The wearable muscle interface devices described herein aregenerally not operative to identify any arbitrary gesture performed by auser. Rather, the wearable muscle interface devices described herein areoperative to identify when a user performs one of a specified set ofgestures, and that specified set of gestures is referred to herein as agesture library. A gesture library may include any number of gestures,though a person of skill in the art will appreciate that theprecision/accuracy of gesture identification may be inversely related tothe number of gestures in the gesture library. A gesture library may beexpanded by adding one or more gesture(s) or reduced by removing one ormore gesture(s). Furthermore, in accordance with the present systems,articles, and methods, a gesture library may include a “rest” gesturecorresponding to a state for which no activity is detected and/or an“unknown” gesture corresponding to a state for which activity isdetected but the activity does not correspond to any other gesture inthe gesture library.

Throughout this specification and the appended claims, infinitive verbforms are often used. Examples include, without limitation: “to detect,”“to provide,” “to transmit,” “to communicate,” “to process,” “to route,”and the like. Unless the specific context requires otherwise, suchinfinitive verb forms are used in an open, inclusive sense, that is as“to, at least, detect,” to, at least, provide,” “to, at least,transmit,” and so on.

The above description of illustrated embodiments, including what isdescribed in the Abstract, is not intended to be exhaustive or to limitthe embodiments to the precise forms disclosed. Although specificembodiments of and examples are described herein for illustrativepurposes, various equivalent modifications can be made without departingfrom the spirit and scope of the disclosure, as will be recognized bythose skilled in the relevant art. The teachings provided herein of thevarious embodiments can be applied to other portable and/or wearableelectronic devices, not necessarily the exemplary wearable electronicdevices generally described above.

For instance, the foregoing detailed description has set forth variousembodiments of the devices and/or processes via the use of blockdiagrams, schematics, and examples. Insofar as such block diagrams,schematics, and examples contain one or more functions and/oroperations, it will be understood by those skilled in the art that eachfunction and/or operation within such block diagrams, flowcharts, orexamples can be implemented, individually and/or collectively, by a widerange of hardware, software, firmware, or virtually any combinationthereof. In one embodiment, the present subject matter may beimplemented via Application Specific Integrated Circuits (ASICs).However, those skilled in the art will recognize that the embodimentsdisclosed herein, in whole or in part, can be equivalently implementedin standard integrated circuits, as one or more computer programsexecuted by one or more computers (e.g., as one or more programs runningon one or more computer systems), as one or more programs executed by onone or more controllers (e.g., microcontrollers) as one or more programsexecuted by one or more processors (e.g., microprocessors, centralprocessing units, graphical processing units), as firmware, or asvirtually any combination thereof, and that designing the circuitryand/or writing the code for the software and or firmware would be wellwithin the skill of one of ordinary skill in the art in light of theteachings of this disclosure.

When logic is implemented as software and stored in memory, logic orinformation can be stored on any computer-readable medium for use by orin connection with any processor-related system or method. In thecontext of this disclosure, a memory is a computer-readable medium thatis an electronic, magnetic, optical, or other physical device or meansthat contains or stores a computer and/or processor program. Logicand/or the information can be embodied in any computer-readable mediumfor use by or in connection with an instruction execution system,apparatus, or device, such as a computer-based system,processor-containing system, or other system that can fetch theinstructions from the instruction execution system, apparatus, or deviceand execute the instructions associated with logic and/or information.

In the context of this specification, a “non-transitorycomputer-readable medium” can be any element that can store the programassociated with logic and/or information for use by or in connectionwith the instruction execution system, apparatus, and/or device. Thecomputer-readable medium can be, for example, but is not limited to, anelectronic, magnetic, optical, electromagnetic, infrared, orsemiconductor system, apparatus or device. More specific examples (anon-exhaustive list) of the computer readable medium would include thefollowing: a portable computer diskette (magnetic, compact flash card,secure digital, or the like), a random access memory (RAM), a read-onlymemory (ROM), an erasable programmable read-only memory (EPROM, EEPROM,or Flash memory), a portable compact disc read-only memory (CDROM),digital tape, and other non-transitory media.

The various embodiments described above can be combined to providefurther embodiments. To the extent that they are not inconsistent withthe specific teachings and definitions herein, all of the U.S. patents,U.S. patent application publications, U.S. patent applications, foreignpatents, foreign patent applications and non-patent publicationsreferred to in this specification and/or listed in the Application DataSheet, including but not limited to U.S. Provisional Patent ApplicationNo. 61/752,226, filed Jan. 14, 2013, U.S. Provisional Patent ApplicationSer. No. 61/768,322, U.S. Provisional Patent Application Ser. No.61/771,500, U.S. Provisional Patent Application Ser. No. 61/857,105,U.S. Provisional Patent Application Ser. No. 61/860,063, U.S.Provisional Patent Application Ser. No. 61/822,740, U.S. ProvisionalPatent Application Ser. No. 61/866,960, U.S. Provisional PatentApplication Ser. No. 61/869,526, U.S. Provisional Patent ApplicationSer. No. 61/874,846, U.S. Provisional Patent Application Ser. No.61/872,569, US Provisional Patent Application Serial No. 61/881,064,U.S. Provisional Patent Application Ser. No. 61/894,263, U.S.Provisional Patent Application Ser. No. 61/903,238, U.S. ProvisionalPatent Application Ser. No. 61/909,786, and/or U.S. Provisional PatentApplication Ser. No. 61/915,338, are incorporated herein by reference,in their entirety. Aspects of 5 the embodiments can be modified, ifnecessary, to employ systems, circuits and concepts of the variouspatents, applications and publications to provide yet furtherembodiments.

These and other changes can be made to the embodiments in light of theabove-detailed description. In general, in the following claims, theterms used should not be construed to limit the claims to the specificembodiments disclosed in the specification and the claims, but should beconstrued to include all possible embodiments along with the full scopeof equivalents to which such claims are entitled. Accordingly, theclaims are not limited by the disclosure.

The invention claimed is:
 1. A wearable muscle interface device that inuse interacts with content displayed on a wearable head-mounted display,the wearable muscle interface device comprising: a plurality of muscleactivity sensors to be worn on an arm of a user, the muscle activitysensors responsive to electrical signals generated by muscles in the armof the user; at least one accelerometer; a processor configured to:generate, based on at least the electrical signals detected by themuscle activity sensors and motion of the arm of the user detected bythe at least one accelerometer, a first interpretation of a gestureperformed by the user, and a second interpretation of the gestureperformed by the user, and generate at least one control signal torequest input from the user to select one of the first interpretation ofthe gesture or the second interpretation of the gesture, wherein thefirst interpretation of the gesture and the second interpretation of thegesture represent different gestures selected by the processor from aset of at least three defined gestures; a transmitter communicativelycoupled to the plurality of muscle activity sensors, wherein in use thetransmitter transmits at least one signal from the wearable muscleinterface device directly to a receiver on the wearable head-mounteddisplay based on the signals detected by the muscle activity sensors;wherein the at least one signal transmitted, in use, from the wearablemuscle interface device directly to the receiver on the wearablehead-mounted display effects at least one interaction with contentdisplayed on the wearable head-mounted display including requesting theinput from the user to select one of the first interpretation of thegesture performed by the user or the second interpretation of thegesture performed by the user.
 2. The wearable muscle interface deviceof claim 1, wherein the processor interprets the signals detected by themuscle activity sensors as gesture, wherein the processor iscommunicatively coupled in between the transmitter and the plurality ofmuscle activity sensors, and wherein the at least one signal that, inuse, is transmitted from the wearable muscle interface device is basedon the gesture interpreted by the processor of the wearable muscleinterface device.
 3. The wearable muscle interface device of claim 1,wherein the wearable head-mounted display includes a processorcommunicatively coupled to the receiver of the wearable head-mounteddisplay, and wherein the at least one signal that, in use, istransmitted from the wearable muscle interface device to the wearablehead-mounted display is interpreted as a gesture by the processor of thewearable head-mounted display.
 4. The wearable muscle interface deviceof claim 1, further comprising a haptic feedback module that in useprovides haptic feedback to the user, the haptic feedback moduleincluding a vibratory motor.
 5. The wearable muscle interface device ofclaim 1, wherein the plurality of muscle activity sensors includes atleast one muscle activity sensor selected from the group consisting of:an electromyographic (EMG) sensor and a mechanomyographic (MMG) sensor.6. The wearable muscle interface device of claim 1, wherein the at leastone accelerometer detects signals generated by motion of the arm of theuser, the at least one accelerometer communicatively coupled to thetransmitter, and wherein in use the at least one signal transmitted fromthe transmitter of the wearable muscle interface device directly to thereceiver on the wearable head-mounted display is based on both thesignals detected by the muscle activity sensors and the signals detectedby the at least one accelerometer.
 7. The wearable muscle interfacedevice of claim 1, wherein the transmitter includes a wirelesstransmitter.
 8. A wearable system that in use provides hands-free accessto and control of a portable electronic display, the wearable systemcomprising: i) a wearable muscle interface device comprising: aplurality of muscle activity sensors to be worn on an arm of a user, themuscle activity sensors responsive to electrical signals generated bymuscles in the arm of the user; at least one accelerometer; a processorconfigured to: generate, based on at least the electrical signalsdetected by the muscle activity sensors and motion of the arm of theuser detected by the at least one accelerometer, a first interpretationof a gesture performed by the user, and a second interpretation of thegesture performed by the user, and generate at least one control signalto request input from the user to select one of the first interpretationof the gesture or the second interpretation of the gesture, wherein thefirst interpretation of the gesture and the second interpretation of thegesture represent different gestures selected by the processor from aset of at least three defined gestures; a transmitter communicativelycoupled to the plurality of muscle activity sensors, wherein in use thetransmitter transmits at least one signal from the wearable muscleinterface device based on the signals detected by the muscle activitysensors; and ii) a wearable head-mounted display comprising: at leastone display screen to be worn on a head of the user, the at least onedisplay screen arranged to be positioned in front of at least one eye ofthe user when worn on the head of the user; a receiver communicativelycoupled to the at least one display screen, wherein in use the receiverdirectly receives the at least one signal transmitted from thetransmitter of the wearable muscle interface device; and a processorcommunicatively coupled to the receiver and to the at least one displayscreen, wherein in use the at least one signal received directly fromthe transmitter of the wearable muscle interface device by the receiverof the wearable head-mounted display effects control of at least onefunction of the wearable head-mounted display including requesting theinput from the user to select one of the first interpretation of thegesture performed by the user or the second interpretation of thegesture performed by the user.
 9. The wearable system of claim 8,wherein the transmitter of the wearable muscle interface device includesa wireless transmitter and the receiver of the wearable head-mounteddisplay includes a wireless receiver.
 10. The wearable system of claim8, wherein the processor interprets the signals detected by the muscleactivity sensors as a gesture, wherein the processor of the wearablemuscle interface device is communicatively coupled in between thetransmitter and the plurality of muscle activity sensors, and whereinthe at least one signal that, in use, is transmitted from the wearablemuscle interface device is based on the gesture interpreted by theprocessor of the wearable muscle interface device.
 11. The wearablesystem of claim 8, wherein the plurality of muscle activity sensorsincludes at least one muscle activity sensor selected from the groupconsisting of: an electromyographic (EMG) sensor and a mechanomyographic(MMG) sensor.
 12. The wearable system of claim 8, wherein theaccelerometer detects signals generated by motion of the arm of theuser, the at least one accelerometer communicatively coupled to thetransmitter, and wherein in use the at least one signal transmitted bythe transmitter of the wearable muscle interface device is based on boththe signals detected by the muscle activity sensors and the signalsdetected by the at least one accelerometer.
 13. A method of using awearable system to achieve hands-free access to and control of aportable electronic display, wherein the wearable system includes awearable muscle interface device, at least one accelerometer, aprocessor, and a wearable head-mounted display, the method comprising:detecting muscle activity corresponding to a physical gesture performedby a user of the wearable system by at least one muscle activity sensorof the wearable muscle interface device; generating, based on at leastthe electrical signals detected by the at least one muscle activitysensor and motion of the arm of the user detected by the at least oneaccelerometer, a first interpretation of a gesture performed by theuser, and a second interpretation of the gesture performed by the user,and generating at least one control signal to request input from theuser to select one of the first interpretation of the gesture or thesecond interpretation of the gesture, wherein the first interpretationof the gesture and the second interpretation of the gesture representdifferent gestures selected by the processor from a set of at leastthree defined gestures; transmitting at least one signal from thewearable muscle interface device by a transmitter of the wearable muscleinterface device based at least in part on the muscle activity detectedby at least one muscle activity sensor of the wearable muscle interfacedevice; receiving the at least one signal directly from the wearablemuscle interface device by a receiver of the wearable head-mounteddisplay; processing the at least one signal by a processor of thewearable head-mounted display and; effecting at least one interactionbetween the user and the wearable head-mounted display by the processorof the wearable head-mounted display based on the processing of the atleast one signal by the processor of the wearable head-mounted displayincluding requesting the input from the user to select one of the firstinterpretation of the gesture performed by the user or the secondinterpretation of the gesture performed by the user.
 14. The method ofclaim 13, further comprising: in response to detecting muscle activitycorresponding to a physical gesture performed by a user of the wearablesystem by at least one muscle activity sensor of the wearable muscleinterface device, processing the detected muscle activity by theprocessor of the wearable muscle interface device, and whereintransmitting at least one signal from the wearable muscle interfacedevice by a transmitter of the wearable muscle interface device based atleast in part on the muscle activity detected by at least one muscleactivity sensor of the wearable muscle interface device includestransmitting at least one signal from the wearable muscle interfacedevice by the transmitter of the wearable muscle interface device basedat least in part on processing the detected muscle activity by theprocessor of the wearable muscle interface device.
 15. The method ofclaim 13, further comprising: detecting motion of the wearable muscleinterface device corresponding to the physical gesture performed by theuser of the wearable system by the at least one accelerometer of thewearable muscle interface device, and wherein transmitting at least onesignal from the wearable muscle interface device by a transmitter of thewearable muscle interface device based at least in part on the muscleactivity detected by at least one muscle activity sensor of the wearablemuscle interface device includes transmitting at least one signal fromthe wearable muscle interface device by the transmitter of the wearablemuscle interface device based on both the muscle activity detected by atleast one muscle activity sensor of the wearable muscle interface deviceand the motion detected by at least one accelerometer of the wearablemuscle interface device.
 16. The method of claim 13, wherein:transmitting at least one signal from the wearable muscle interfacedevice by a transmitter of the wearable muscle interface device includeswirelessly transmitting at least one signal from the wearable muscleinterface device by a wireless transmitter of the wearable muscleinterface device; and receiving the at least one signal directly fromthe wearable muscle interface device by a receiver of the wearablehead-mounted display includes wirelessly receiving the at least onesignal directly from the wearable muscle interface device by a wirelessreceiver of the wearable head-mounted display.
 17. The method of claim13, wherein a haptic feedback module provides haptic feedback to theuser.
 18. The method of claim 17, wherein the haptic feedback moduleincludes a vibratory motor.
 19. The method of claim 13, wherein the atleast one muscle activity sensor comprises a muscle activity sensorselected from the group consisting of: an electromyographic (EMG) sensorand a mechanomyographic (MMG) sensor.
 20. The method of claim 13,wherein the transmitter of the wearable muscle interface device includesa wireless transmitter and the receiver of the wearable head-mounteddisplay includes a wireless receiver.